Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix/903 portfolio linking sub directories #950

Merged
merged 10 commits into from
Jan 19, 2024
Merged

Fix/903 portfolio linking sub directories #950

merged 10 commits into from
Jan 19, 2024

Conversation

sambles
Copy link
Contributor

@sambles sambles commented Jan 18, 2024 •
edited
Loading

Fixed portfolio linking from sub-directories

  • Fixed #903 shared-fs file linking removes sub-directories when storing new file reference.
  • Fixed S3 bucket storage for V2 workers
  • Added S3 storage testing for both V2 and V1 workers

@sambles sambles linked an issue Jan 18, 2024 that may be closed by this pull request
File linking OED from sub-directories fails to link inside workers #903
Closed
@github-advanced-security
Copy link

This pull request sets up GitHub code scanning for this repository. Once the scans have completed and the checks have passed, the analysis results for this pull request branch will appear on this overview. Once you merge this pull request, the 'Security' tab will show more code scanning analysis results (for example, for the default branch). Depending on your configuration and choice of analysis tool, future pull requests will be annotated with code scanning analysis results. For more information about GitHub code scanning, check out the documentation.

@sambles
Copy link
Contributor Author

sambles commented Jan 18, 2024 •
edited
Loading

Bug - S3 Storage - V1 worker check

[2024-01-18 13:44:55,064: ERROR/ForkPoolWorker-1] Task run_analysis[60027fe2-9da6-4e5c-8437-2612d8ec51b4] raised unexpected: TypeError('AwsObjectStore._fetch_file() takes from 2 to 3 positional arguments but 4 were given')
Traceback (most recent call last):
  File "/home/worker/.local/lib/python3.10/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
  File "/home/worker/.local/lib/python3.10/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
  File "/home/worker/src/model_execution_worker/tasks.py", line 337, in start_analysis_task
    output_location, traceback_location, log_location, return_code = start_analysis(
  File "/home/worker/.local/lib/python3.10/site-packages/oasislmf/utils/log.py", line 123, in wrapper
    result = func(*args, **kwargs)
  File "/home/worker/src/model_execution_worker/tasks.py", line 401, in start_analysis
    filestore.extract(input_archive, oasis_files_dir)
  File "/home/worker/src/model_execution_worker/storage_manager.py", line 207, in extract
    local_archive_path = self.get(
  File "/home/worker/src/model_execution_worker/storage_manager.py", line 300, in get
    return self._fetch_file(reference, output_path, subdir)
TypeError: AwsObjectStore._fetch_file() takes from 2 to 3 positional arguments but 4 were given

@sambles sambles self-assigned this Jan 19, 2024
@sambles sambles merged commit 1b8a36e into main Jan 19, 2024
25 checks passed
@sambles sambles deleted the fix/903-portfolio-linking-sub-directories branch January 19, 2024 12:44
@awsbuild awsbuild added this to the 2.3.0 milestone Feb 6, 2024
@sambles sambles mentioned this pull request Feb 22, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@sambles sambles

Labels
bug kubernetes
Projects
Oasis Dev Team Tasks
Status: Done
Milestone
2.3.0
Development

Successfully merging this pull request may close these issues.

File linking OED from sub-directories fails to link inside workers
2 participants
@sambles @awsbuild